Repository navigation
test(usage): anchor persistent fixtures to a classified-persistent base - #8338
huangruiteng merged 2 commits into
Conversation
The two persistent-route tests built their fixtures under a CWD-relative .local directory. From a checkout or worktree under a temporary root — a legitimate contributor layout, and on macOS /tmp resolves to /private/tmp — the production classifier correctly calls that fixture temporary, because it also weighs the running module's own location (fileURLToPath(import.meta.url) is one of its candidates): a LoopX running from a throwaway checkout suppresses usage ping everywhere. The tests then failed deterministically and read as main regressions. The fixtures now anchor to a base the classifier itself calls persistent (homedir first, then the checkout), and skip with that stated reason when no candidate qualifies — which is exactly the case of a temp-root checkout, where the persistent-route behavior genuinely cannot be constructed. Verified from both layouts: 3/3 pass from a persistent checkout (unchanged), 1 pass + 2 reasoned skips from a /private/tmp worktree where 2 hard failures stood before. Signed-off-by: Lihua <1017343802@qq.com>
|
CI attribution for |
loopx-agent
left a comment
loopx-agent
left a comment
There was a problem hiding this comment.
Reviewer: model_agent | gpt-6.1-sol | OpenAI | runtime_reported | reasoning_effort=max
动机 / Motivation
维护使用统计隔离测试的人，在临时目录中检出源码并运行测试时会遇到这个问题。分类规则把源码模块所在目录也计入临时环境；旧测试仍断言存在持久环境，因此临时检出会失败。新写法尝试选择持久目录，无候选时跳过相关测试。临时检出不再误报，但正常检出中分类器错误地把所有路径判为临时时，新增跳过也会掩盖这个回归。范围仅是已有测试的 fixture 位置和跳过条件，不修改遥测资格、披露、网络传输或产品设置。
改动思路 / Approach
temporaryUsageEnvironment 是本次要保护的分类器；它读取状态路径、home、临时根以及自身模块的真实路径。新增 persistentFixtureBase 先用这个分类器筛选 home 与 CWD，再由空结果触发整项 t.skip。这样把“环境确实无法提供持久模块”和“分类器错误地拒绝持久路径”合并成同一种跳过，形成自证式的测试前置条件。
具体改动 / Changes
关键内容讲解
准确审阅 head 8d9b017afa0a361f476e667fa44e776c697b8f5d，仅测试文件 +24/-6。第 20–24 行的 helper 复用生产分类器选目录；第 73–76 行和第 94–97 行在任何持久行为或路径边界断言之前跳过。原有临时传输隔离测试不变，t.after 清理保留。
规范依据：docs/development/good-smokes.md，固定修订 21013c0c113445b71a30f6ce06ab61455b56f1f2。其未编号表格行 What is the oracle? 在此 head 为 not_met：是否执行反例测试由同一个被测分类器决定。期望应来自独立审阅的路径规则，而非被测函数是否愿意进入断言。
同一 Node、同一生产源码和相同持久目录条件下，独立比较不可变 base 与 head；mutation 只在私有一次性源码副本中把 temporaryUsageEnvironment 加入 return true，没有改作者分支：
| 源码与测试组合 | 结果 |
|---|---|
| 原始 base / 原始 head | 各 PASS：3 passed、0 skipped |
| 同一“始终临时”mutation + base 测试 | 正确拒绝：1 passed、2 failed、exit 1 |
| 同一 mutation + 此 head 测试 | 漏检：1 passed、2 skipped、exit 0 |
| 此 head，模块实际位于系统临时目录 | 1 passed、2 skipped、exit 0 |
相邻七个 usage 测试文件共 70 项 PASS；base/head 的 npm run typecheck:control-plane 均 PASS；此 head 原生 quick premerge 的 4 项直接检查 PASS。负例验证失败的是“必须拒绝分类回归”的要求，普通绿测不能抵销它。所有传输资格测试只使用数字回环地址或拒绝路径，没有向真实统计端点发请求，也没有查询远端 CI。
对主干的风险 / Risks
[P2] 不要让被测分类器决定是否跳过自己的持久分支（第 22 行）。 一旦正常路径被错误判为临时，两个候选都被拒绝，随后两项测试直接跳过。除了 notice-before-default-on 的保护，原本对 scratch 与 scratch-persistent 的组件边界区分也一起失去检测。实际 mutation 已证明旧测试失败、新测试成功退出；这项结论限定于这个被改文件，没有声称整个仓库都检测不到该 mutation。用独立环境事实判断“模块确实位于临时根下”的不可用条件；正常持久检出必须断言至少存在持久候选，不能由被测分类器返回 true 来豁免自身。把不依赖持久候选的临时路径与 alias 断言保留为可执行覆盖。
保留现有文件即可，无需新增一套测试框架或改生产策略。已扫描其它 usage 测试的生产调用及同作者 #8336、#8340；这些主题不同，没有已验证的重复铺设或重复违规证据，不提出贡献限制。
我的整体评价 / Verdict
REQUEST_CHANGES。 临时检出的兼容目标合理，但正常检出丢失了原有独立失败信号，当前修复边界尚未交付。long_horizon 与 user_experience 均为 not_applicable：本 diff 不改变实际 Goal 续跑、用户设置或产品界面，缺陷在测试回归保护。视觉检查为 not_applicable，因为 tests/control_plane_ts/usage_statistics_environment.test.ts 只涉及测试 fixture；未验证范围包括跨平台 Windows 执行，未宣称产品临时环境策略有新回归。没有新增共享语义词汇、生产默认值或持久格式兼容分支。修复后应同时保留“真实临时检出可解释跳过”和“正常检出 mutation 必须失败”两个对照。没有合并授权。
English verdict: REQUEST_CHANGES — classifier-driven skips hide persistent-path regressions; derive availability independently and retain a failing regression oracle.
Review follow-up: choosing the fixture base with the classifier under test made the oracle self-referential — a regressed classifier that rejects every path would turn both scenarios into silent skips instead of the failed assertions that catch the regression. The base selection now restates the documented temporary-root rules independently (including the classifier's own module-location dimension, for which the test file stands in), so: a persistent checkout proceeds and a classifier regression fails the assertions; a temp-root checkout skips with the documented reason, where the all-temporary answer is the correct behavior. Verified three ways on this head: persistent checkout 3/3 pass; a /private/tmp worktree 1 pass + 2 reasoned skips; and a mutation probe that forces the classifier to always report temporary now yields 1 pass + 2 FAILED (previously masked as skips). Signed-off-by: Lihua <1017343802@qq.com>
LIHUA919
left a comment
LIHUA919
left a comment
There was a problem hiding this comment.
Addressed on new head 1aa8eb73f: the fixture base is now selected by an INDEPENDENT path oracle — the documented temporary-root rules restated in the test (roots, env vars, physical resolution, and the classifier's own module-location dimension, for which the test file stands in) — not by the classifier under test. Your mutation case is reproduced and now caught: forcing temporaryUsageEnvironment to always return true yields 1 pass + 2 FAILED (previously masked as 2 skips, exit 0). Full matrix on this head: persistent checkout 3/3 pass; /private/tmp worktree 1 pass + 2 reasoned skips (the documented module-location behavior). English verdict: the oracle no longer consults the function under test; the regression fails loudly.
loopx-agent
left a comment
loopx-agent
left a comment
There was a problem hiding this comment.
Reviewer: model_agent | gpt-6.1-sol | OpenAI | runtime_reported | reasoning_effort=max
动机 / Motivation
维护使用统计隔离测试的人，在临时目录检出源码并运行测试时会遇到这个问题。生产规则把模块所在目录也作为临时环境依据，原测试却要求临时检出能构造持久环境；此 PR 选择持久 fixture，并在模块确实处于临时根时解释跳过。上轮由被测分类器决定跳过，会掩盖分类器回归；当前 head 改为独立路径判断，已恢复这个失败信号。范围仅是已有测试的 fixture 和环境可用性，不修改遥测策略、披露、传输或产品设置。
改动思路 / Approach
当前 independentlyTemporary 从操作系统临时根、显式临时环境变量、物理路径和组件边界判断可用性，不调用被测的 temporaryUsageEnvironment。persistentFixtureBase 先判断测试模块的位置，再选择 home 或 CWD。只有环境事实无法提供持久分支时才跳过；正常持久检出必须进入断言，由生产分类器的实际结果决定测试成功或失败。
继续使用现有三个测试和真实数字回环传输，清理仍由 t.after 负责。新增约五十行独立规则有维护成本，但从同一个生产分类器导出跳过理由会重新引入已证实的漏检；当前独立 oracle 边界有必要。将来规则扩展应单独审阅期望，不能仅复制新输出刷新测试。这不是新的产品分类 authority。
具体改动 / Changes
关键内容讲解
准确 head 1aa8eb73fab7500274609ffd7c9a858feb16e162，相对 21013c0c113445b71a30f6ce06ab61455b56f1f2 只改一个测试文件 +62/-7。相对上轮 8d9b017afa0a361f476e667fa44e776c697b8f5d 的差异为 +49/-12，核心是替换自证式 fixture 筛选。independentlyTemporary 处理缺失尾路径、物理 alias、临时根及组件边界；persistentFixtureBase 让真实临时模块得到明确跳过。原三项测试的网络隔离、notice-before-default-on、scratch 与 scratch-persistent 区分及 symlink 断言保留。
规范依据：docs/development/good-smokes.md，固定修订 21013c0c113445b71a30f6ce06ab61455b56f1f2。其未编号表行 What is the oracle? 为 implemented：可用性判断不读取被测函数结果；What boundary is exercised? 为 implemented：保留真实 inspect/configure/observe 和数字回环 HTTP；Is the assertion durable? 为 implemented：验证的是资格、状态和零请求/发送正例。另已按同修订 docs/reference/usage-ping.md 的 Disclosure and precedence 核对临时模块、物理 alias、临时 CWD 单独不构成临时安装及显式数字回环例外。
同一 Node 24.21.0/macOS、同一未变生产代码，独立对照不可变 base、上轮和当前测试：
| 条件 | 结果 |
|---|---|
| 正常持久检出，原 base / 当前 head | 各 PASS：3 passed、0 skipped |
| 分类器始终返回临时，base 测试 | 1 passed、2 failed，exit 1 |
| 同一 mutation，上轮测试 | 1 passed、2 skipped，exit 0，保留历史漏检 |
| 同一 mutation，当前 head 测试 | 1 passed、2 failed，exit 1，正确检出 |
| 当前 head，模块实际位于 OS 临时目录 | 1 passed、2 skipped，说明可用性原因 |
mutation 只改独立一次性源码副本，未改作者 checkout。相邻七个 usage 测试文件共 39 项 PASS。初次 base/head 均缺 tsc，保留 setup 失败；安装原锁定依赖后，二者 npm run typecheck:control-plane 均 PASS。源码标准 premerge 的 4 项直接检查 PASS，目录匹配没有选择额外 smoke；没有把零项当额外覆盖。没有查询远端 CI。
对主干的风险 / Risks
未发现当前 head 的阻断问题。最强反例是“生产分类器拒绝所有持久路径，而测试看似成功”；当前独立 oracle 已使两项断言失败，恢复旧测试的保护。真实临时检出仍跳过相关持久分支，不把正确抑制误报为回归。第一项全通道隔离和发送正例继续执行。fixture 仅使用数字回环地址、注入传输或本地拒绝地址，未向真实收集端点发送。
规则重复带来日后漂移风险，正常持久环境的 mutation 对照应保留。Windows 分支未实际运行，Node 最低版本也未在本轮复验；这些保留为平台资格限制，没有声称产品路径或整仓库已获完整资格。同作者 #8336/#8340 的主题及现有 usage 覆盖已核对，不新增同形 smoke，不建议贡献限制。
我的整体评价 / Verdict
APPROVE。 该维护切片已交付：可解释的真实临时检出跳过和正常检出的独立失败信号同时成立。long_horizon 与 user_experience 为 not_applicable：不改变 Goal 续跑或产品交互；视觉检查为 not_applicable，因为 tests/control_plane_ts/usage_statistics_environment.test.ts 只改变测试 fixture，没有渲染变更。未验证范围为 Windows、最低 Node 和完整产品浏览器旅程。没有新增共享状态词汇、生产默认值或持久兼容分支；保留现有测试是当前最小边界。批准不代表合并授权，未合并。
English verdict: APPROVE — head 1aa8eb7 restores an independent regression oracle while retaining explained skips for actual temporary checkouts; platform qualification remains scoped.
Closes #8337.
Problem
The two persistent-route tests in
tests/control_plane_ts/usage_statistics_environment.test.tsbuilt their fixtures under a CWD-relative.localdirectory. From a checkout or git worktree under a temporary root — a legitimate contributor layout, and on macOS/tmpresolves to/private/tmp— the production classifier correctly classifies that fixture as temporary, because it also weighs the running module's own location (fileURLToPath(import.meta.url)is one of its candidates): a LoopX running from a throwaway checkout suppresses usage ping everywhere. The two tests then failed deterministically (expected 'notice_required', actual 'temporary_path') and read as main regressions.Change
persistentFixtureBase(): homedir first, then the checkout), so the tests exercise relative eligibility rather than the checkout's location.Validation
/private/tmpworktree (the failing layout from [Bug]: usage_statistics_environment tests fail when the checkout lives under a temp root (macOS /private/tmp worktrees) #8337): 1 pass + 2 reasoned skips, where 2 hard failures stood before.npm run typecheck:control-planeclean; exact-pathloopx checkboundary scan clean for the changed file (the check's registry-goal errors are pre-existing global-registry state, unrelated to this diff);canary premerge --from-git-diff --git-diff-base origin/mainpassed with 0 failures / 0 warnings.